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Abstract 

Prior research has found that financial investments in North Carolina’s early childhood 
education programs—Smart Start and NC Pre-K—generated positive effects on student 
achievement in reading and mathematics through eighth grade (Bai et al., 2020). The current 
study examined if these effects were moderated by two dimensions of educational opportunity in 
NC public school districts, as measured by (1) the average level of academic achievement among 
third-grade students in the school district and (2) the rate of growth in academic achievement 
among students in the school district as they progressed from third to eighth grade. The Smart 
Start effect on eighth grade reading achievement was larger in school districts with higher levels 
of average achievement. Also, the NC Pre-K effect on eighth grade reading achievement was 
smaller in school districts with higher rates of achievement growth. 

Keywords: Child care; early childhood education; school quality; reading; mathematics; 


long-term effects. 
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Introduction 

Public investments in children’s early skill development can result in substantial benefits 
to society, especially if those investments target children from families who experience 
economic insecurity (Heckman, 2006). These benefits can include increased academic 
achievement and educational attainment, improved health outcomes, greater earning potential in 
adulthood, and reduced criminal behavior (Yoshikawa et al., 2013). Today, early childhood 
education (ECE) is widely recognized as a proven strategy to promote children’s early skill 
development, with benefits that far outweigh their costs. For example, highly effective programs 
have generated returns on investments that range from three to seven dollars saved for every 
dollar invested (Yoshikawa et al., 2013). 

Despite the obvious importance of investing in early childhood skill development, 
children also benefit from later investments in their skill development throughout childhood. 
Indeed, a wealth of evidence suggests that the quality of educational opportunity in schools can 
significantly promote children’s skill development during childhood and adolescence (Eccles & 
Roeser, 2010). An important question not yet fully resolved is how early investments can interact 
with later investments in children’s skill development. In particular, understanding if the quality 
of subsequent school environments moderates the long-term impacts of ECE throughout the 
formal schooling years is a topic of interest to both policy and scientific audiences alike (Phillips 
et al., 2017).! 

The quality of the schooling environment has been hypothesized as a primary factor in 


determining whether the impacts of ECE persist in the long run (Bailey et al., 2017; Phillips et 


‘Tn the current study, we refer to ECE program effects on eighth grade outcomes as “long-term” effects because 
children have entered adolescence by eighth grade. Alternatively, use of the term “short-term” effects often pertains 
to effects on children’s outcomes at the end of the preschool period and effects on child outcomes during elementary 
school are often referred to as medium-term effects (e.g., Unterman & Weiland, 2020). 
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al., 2017). Yet, there remains some debate regarding the mechanisms by which subsequent 
education quality will moderate the long-term impacts of ECE. There are currently two leading 
hypotheses: dynamic complementarity and dynamic substitutability (Bailey, Duncan, et al., 
2020). Dynamic complementarity implies that ECE program impacts will be maintained and may 
even be enhanced in the context of higher-quality educational environments during school. 
Alternatively, dynamic substitutability implies that ECE program impacts will be diminished in 
the context of higher-quality educational environments during school. The existing research base 
is generally mixed and provides no clear consensus regarding the role of subsequent school 
environments in differentiating the impacts of ECE (Bailey, Jenkins, et al., 2020). Further 
research is needed to test these alternative hypotheses. 

Previous research has found that financial investments in North Carolina’s two, flagship 
ECE programs—Smart Start and NC Pre-K—produced positive effects on student academic 
achievement through eighth grade (described in greater detail below) (Bai et al., 2020).? In the 
current study, we examined whether these long-term effects were moderated by the quality of 
educational opportunity that children subsequently experienced in NC public school districts— 
focusing on program effects on student achievement in eighth grade. We used a linked 
administrative dataset including more than one million children who were born in NC across an 
18-year period. Following Reardon’s (2018) operationalization of “educational opportunity,” we 
measured the quality of educational opportunity in NC public school districts along two 
dimensions: (1) the average level of academic achievement among third-grade students in the 


school district and (2) the rate of growth in academic achievement among students in the school 


? The NC Pre-K program was previously known as the More at Four program. In 2011, the NC General Assembly 
transferred the existing More at Four program from the Department of Public Instruction to the Division of Child 
Development and Early Education in the Department of Health and Human Services and renamed it to the NC Pre-K 
program. 
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district as they progressed from third to eighth grade. We then tested for moderation of Smart 
Start and NC Pre-K program funding effects by the quality of educational opportunity in NC 
public school districts. The results of this study offer implications for the alignment of high- 
quality educational programming across the preschool and school-age periods. 

North Carolina’s ECE programs 
Smart Start 

The Smart Start program was established in 1993 by the NC General Assembly to create 
a comprehensive and integrated system of developmental services for children (and their 
families) from birth to age 4. The Smart Start program provides services in three main areas: 
child care (focusing on affordability, availability, and quality), family support, and child health. 
Smart Start is administered by the NC Partnership for Children, which awards funds to local 
Smart Start partnerships to provide program services in each of NC’s 100 counties. Local 
decision-making is a hallmark of the Smart Start initiative as the activities of each local 
partnership are guided by a board of community members who select the services that best meet 
the needs of children and families in their community (North Carolina Partnership for Children, 
2019). Therefore, the scope and availability of local Smart Start services can vary across counties 
and over time. 

While some local Smart Start services are universally available to all children and 
families within communities, other services are targeted to children from lower-income families. 
For example, child care subsidies constitute a large portion of Smart Start expenditures and 
eligibility for the sliding fee scale is limited to families earning at or below 75% of the state’s 
median income (North Carolina Partnership for Children, 2019). At the same time, efforts to 


improve the quality of child care programs in communities may benefit all children, regardless of 
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whether or not they receive a child care subsidy. During the time period of Smart Start 
implementation considered in our study, the quality of local child care centers sponsored by 
Smart Start was found to improve on a wide range of quality metrics (Bryant et al., 2003). For 
example, the percent of children attending high-quality child care programs (4- and 5-star 
programs rated on a 5-star scale) doubled from 33% in 2001 to 63% in 2010 (the last program 
year considered in our study) (North Carolina Partnership for Children, 2010). 
NC Pre-K 

Established by NC General Assembly in 2001, NC Pre-K is a state-funded educational 
program for eligible 4-year-old children, designed to enhance their school readiness skills prior 
to kindergarten entry. During our study period between 2001 and 2010, the NC Pre-K program 
was administered by the NC Department of Public Instruction (DPJ) and then, in 2011, the NC 
General Assembly transferred the program to the Division of Child Development and Early 
Education (DCDEE) in the NC Department of Health and Human Services.’ The state 
administrative agency awards funds to local contractors to provide NC Pre-K programming in 
each of NC’s 100 counties—funding qualified child care centers to serve eligible children in a 
variety of classroom-based educational settings, including public schools, Head Start, and private 
child care centers (both for-profit and nonprofit). While NC was among the latter half of states to 
adopt a public pre-kindergarten program (Cohen-Vogel et al., 2020), NC Pre-K grew quickly to 
become an established program operating at scale. During the 2009-10 program year (the last 
program year considered in our study), 24% of the state’s 4-year-old population participated in 
the program (N = 31,197; Barnett et al., 2010) and that figure has remained relatively consistent 


over time (e.g., 24% served in 2019; Friedman-Krauss et al., 2020). 


3 Previously known as the More at Four program, the program was renamed as the NC Pre-K program when it was 
transferred from the NC DPI to DCDEE. 
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NC Pre-K program guidelines allow 4-year-old children to receive NC Pre-K funding if 
their family has a gross family income at or below 75% of the state median income, or if the 
child has at least one of the following factors: educational or developmental delay, an identified 
disability, a chronic health condition, or limited English proficiency (Barnett et al., 2010).* 
Funding for NC Pre-K is targeted to children who meet the eligibility criteria. Nonetheless, 
aspects of the program’s design may lead to benefits for other children in communities— 
particularly those children who do not directly receive NC Pre-K funding, but still participate in 
a NC Pre-K funded child care center through some other funding means. For example, NC Pre-K 
contractors are expected to meet a variety of program standards designed to ensure a high- 
quality, classroom-based educational experience for all children enrolled at their local sites— 
both NC Pre-K funding recipients and non-recipients. The state administrative agency regularly 
implements quality monitoring and improvement initiatives to promote uniformity in NC Pre-K 
program services across the state, to the extent possible. Therefore, all children in communities 
may benefit from increases in ECE quality because of NC Pre-K. Moreover, NC Pre-K has long 
maintained high standards for program quality. During the 2009-10 program year, the NC Pre-K 
program met all 10 of the quality benchmarks established by the National Institute of Early 
Education Research—tepresenting an increase from 7 quality benchmarks met during the 2002- 
03 program year (Barnett et al., 2010; Barnett et al., 2003). 

Background Literature 
The Long-Term Impacts of ECE 
A robust body of evidence suggests that ECE programming can have positive, short-term 


impacts on children’s academic and social-behavioral readiness to succeed in elementary school 


4 After our study period, NC Pre-K program eligibility was extended to children with a parent serving in the 
military. 
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(Camilli et al., 2010). Research has also found that ECE programming can have long-term 
impacts on educational outcomes throughout the school-age years and into adulthood (McCoy et 
al., 2017). Early evidence comes from random assignment studies of two model ECE 
programs—the Abecedarian and Perry Preschool programs—which both documented positive 
effects on academic achievement outcomes at school entry and throughout the school-age years, 
as well as positive effects on overall educational attainment in adulthood; including higher rates 
of high school graduation and enrollment in postsecondary programs (Campbell et al., 2002; 
Schweinhart et al., 1985). 

More recent studies have also documented evidence of the positive, long-term effects of 
modern day ECE programs operating at scale. This includes a series of quasi-experimental 
studies examining the long-term effects of financial investments in the rollout of the Smart Start 
and NC Pre-K programs (Bai et al., 2020; Dodge et al., 2016; Ladd et al., 2014; Muschkin et al., 
2015). These effects include improvements to student achievement in reading and mathematics 
as well as reductions in special education placements and grade retention—effects that were 
evident in third grade and continued to persist into eighth grade. These findings were drawn from 
a research design that leveraged between-county and within-county variation in the rollout of 
funding for Smart Start and NC Pre-K across NC’s 100 counties over multiple years (a 
generalization of the difference-in-differences design). These effects were measured at a 
population-wide level and could have been conferred through direct effects on former program 
participants as well as spillover effects to non-participants. 

Using similar research designs with county-level variation in ECE program rollout, 
several studies have documented evidence of the positive, long-term, and population-wide 


effects on educational outcomes based on the rollout of the federal Head Start preschool program 
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(Bailey et al., 2021; Johnson & Jackson, 2019; Kose, 2021; Thompson, 2018) and Georgia’s 
universal pre-kindergarten program (Fitzpatrick, 2008), as well as the rollout of targeted 
preschool programming for low-income children in Denmark (Rossin-Slater & Wiist, 2020). 
Positive long-term effects of Head Start have also been documented based on studies of sibling 
comparisons (Currie & Thomas, 1995; Garces et al., 2002; Johnson & Jackson, 2019; Pages et 
al., 2020) and regression discontinuity research designs (Carneiro & Ginja, 2014; Ludwig & 
Miller, 2007). A majority of the aforementioned studies focused on program effects related to 
educational attainment outcomes in adulthood, such as high school graduation and overall years 
of schooling. However, two of these studies complement previous research on Smart Start and 
NC Pre-K program funding by documenting positive effects of ECE program funding on 
academic achievement outcomes during the school-age period (1.e., reading and math test scores; 
Fitzpatrick, 2008; Kose, 2021). 

A separate body of research has used treatment-comparison group designs to contrast 
ECE and non-ECE program participants. Research in this area offers more extensive evaluation 
of ECE program effects on educational outcomes during the school-age period. Three random- 
assignment studies of modern day ECE programs operating at scale have documented positive 
short-term effects on students’ school entry skills—including studies of the federal Head Start 
preschool program (Puma et al., 2010), Tennessee’s state-funded pre-kindergarten program 
(Lipsey et al., 2018), and NC’s NC Pre-K Program (Peisner-Feinberg et al., 2019). However, all 
of these studies document substantial fadeout in effects on academic achievement outcomes 
during the early elementary grades (Lipsey et al., 2018; Peisner-Feinberg et al., 2020; Puma et 
al., 2012). Surprisingly, the random-assignment study of Tennessee’s state-funded pre- 


kindergarten program documented negative effects on academic achievement outcomes in the 
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early elementary grades—after finding initially positive effects at the end of preschool (Lipsey et 
al., 2018). Nonetheless, it is possible for ECE programming to demonstrate positive effects on 
longer-term educational outcomes, even if diminished or null effects on short-term academic 
achievement are observed (Bailey, Duncan, et al., 2020). 

Studies based on quasi-experimental, treatment-comparison group contrasts also provide 
strong evidence of short-term ECE program effects on academic achievement outcomes at school 
entry (Barnett et al., 2018; Gormley et al., 2008; Peisner-Feinberg & Schaaf, 2011; Peisner- 
Feinberg et al., 2014; Weiland & Yoshikawa, 2013). Still, evidence of longer-term impacts on 
academic achievement outcomes is somewhat mixed in these studies—including evidence of 
fadeout in achievement effects documented during the early elementary grades (Bassok et al., 
2019; Hill et al., 2015; Weiland et al., 2019) as well as persistent, positive effects on 
achievement outcomes during elementary school (Bassok et al., 2019; Hill et al., 2015), middle 
school (Gormley et al., 2018; Phillips et al., 2016), and high school (Barnett & Jung, 2021). 
Several of these quasi-experimental studies also provide evidence of ECE program effects on 
broader measures of educational attainment outcomes during the school-age period (Gormley et 
al., 2018; Gray-Lobe et al., 2021; Phillips et al., 2016), and even into college (Gray-Lobe et al., 
2021). The effects of NC Pre-K have also been investigated in a series of quasi-experimental 
studies based on treatment-comparison group contrasts—documenting positive effects on 
academic achievement outcomes at school entry (Peisner-Feinberg & Schaaf, 2011) as well as 
positive effects on academic achievement and educational attainment outcomes during the early 
elementary grades, through third grade (Peisner-Feinberg et al., 2017; Peisner-Feinberg & 


Schaaf, 2010, 2011). 
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In sum, a robust body of evidence demonstrates that ECE programming can lead to 
improved academic achievement in the short term. Additionally, these effects can persist in the 
long term and may even lead to improvements in other educational attainment outcomes. This 
evidence is mainly drawn from random assignment studies of two model ECE programs 
implemented in the 1970’s as well as quasi-experimental studies of modern day ECE programs 
being implemented at scale. Several random assignment studies of modern day ECE programs 
document initially positive effects at school entry and subsequent fadeout in effects on academic 
achievement outcomes. However, it is not clear if and how the quality of subsequent school 
environments play a role in determining whether ECE program effects persist in the long run. 
The Long-Term Impacts of ECE and Subsequent School Quality 

There may be certain conditions under which ECE program impacts are more likely to 
persist in the long run. Moreover, these conditions may be complex and multifaceted, and their 
influence may vary across interventions and outcomes. One hypothesis suggests that ECE 
program impacts will be maintained or even enhanced in the context of higher-quality 
educational environments during school (Bailey et al., 2017; Cunha et al., 2006; Phillips et al., 
2017). Commonly referred to as dynamic complementarity, this hypothesis is rooted in an 
economic theory of human capital investments and suggests that “early investments are not 
productive if they are not followed up by later investments” across the lifespan (Cunha et al., 
2006, p. 698). In the education research literature, the sustaining environments hypothesis 
specifically identifies high-quality school environments as the subsequent investment that is 
needed to maintain the long-term impacts of ECE (Bailey et al., 2017). The fundamental premise 
of these complementary hypotheses rests on the idea that “skills beget skills” or “abilities beget 


abilities” —that skills acquired during an earlier stage of development persist into later stages of 
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development, and enhance the opportunity to acquire more advanced skills at later stages of 
development (Cunha et al., 2006, p. 703). Moreover, ECE program impacts on children’s early 
skill development will carry over into long-term impacts on later skill development if higher- 
quality schooling enables children to further build on the early skills they gained through ECE 
and acquire more advanced skills during school. Through dynamic complementarity, the 
combination of ECE program exposure and high-quality schooling will lead to the highest skill 
attainment for children. 

Alternatively, dynamic substitutability suggests that ECE program impacts will be 
diminished in the context of higher-quality educational environments during school, because the 
benefits of higher-quality schooling will substitute for the benefits of ECE program exposure 
(Bailey, Duncan, et al., 2020). Under these conditions, higher-quality schooling will lead to the 
highest skill attainment for all children regardless of ECE program exposure, as higher-quality 
school environments boost the skill development of children who missed out on ECE and enable 
them to catch up to their peers with previous ECE program exposure (Abenavoli, 2019). 
Dynamic substitutability implies that ECE program impacts will also be maintained in the 
context of lower-quality educational environments during school, as ECE program exposure 
supports children’s continued skill development while compensating for or buffering children 
against subsequent lower-quality educational environments (Abenavoli, 2019; Bailey, Duncan, et 
al., 2020). For example, Abenavoli (2019) describes how high-quality ECE programming may 
promote resilience among children who subsequently enroll in lower-quality school 
environments—which is consistent with evidence that higher-quality child care can compensate 
for or buffer children against the negative impacts of lower socioeconomic status, including 


lower family income (McCartney et al., 2007). Importantly, dynamic substitutability only 
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implies that children with ECE program exposure will be better off relative to children without 
program exposure in lower-quality school environments (i.e., a positive impact of ECE), while 
all children will attain higher skills in higher-quality school environments regardless of ECE 
program exposure (Abenavoli, 2019; Bailey, Duncan, et al., 2020). Statistically, support for the 
dynamic substitutability hypothesis would be evidenced by a negative coefficient on the 
interaction between ECE program exposure and subsequent schooling quality; suggesting a 
counterpoint to the dynamic complementarity hypothesis which would be evidenced by a 
positive interaction coefficient between ECE and subsequent schooling (Bailey, Jenkins, et al., 
2020). 

There are generally mixed findings across studies that have examined how subsequent 
school contexts moderate the long-term effects of ECE (see review by Bailey, Jenkins, et al., 
2020). With regard to financial investments in ECE programming, Johnson and Jackson (2019) 
found evidence in support of dynamic complementarity, whereby the long-term benefits of Head 
Start investments were larger in the context of subsequent higher investments in public 
schooling. Other studies based on ECE treatment-control group designs also provide support for 
dynamic complementarity. These studies considered a variety of subsequent education quality 
measures, including teacher quality (Swain et al., 2015), school-wide academic achievement 
(Ansari & Pianta, 2018; Carr et al., 2021; Unterman & Weiland, 2020; Zhai et al., 2012), and 
school-wide growth in academic achievement (Carr et al., 2021; Pearman et al., 2020; Unterman 
& Weiland, 2020). One study in particular examined the effects of NC Pre-K program 
participation on child outcomes at the end of kindergarten and found that program effects on 
child language and working memory skills were only evident in higher-quality elementary 


schools as measured by school-wide academic proficiency and academic growth, respectively 
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(Carr et al., 2021). However, similar effects on child literacy and math outcomes were not 
evident in that study. 

Alternatively, several studies of ECE program participation provide support for the 
dynamic substitutability hypothesis of diminished ECE impacts in higher-quality school contexts 
(Bierman et al., 2014; Curenton et al., 2015; Magnuson et al., 2007). Perhaps most prominently, 
Magnuson et al. (2007) found that positive effects of ECE participation on academic 
achievement in elementary school were diminished in the context of smaller class sizes and more 
time spent in reading instruction, but ECE effects were maintained during elementary school for 
students in larger classes with less reading instruction (i.e., lower-quality environments). 

The Quality of Educational Opportunity in Public Schools 

In the current study, we sought to index the quality of educational opportunity in NC’s 
public schools. We follow Reardon (2018) who defined and measured two dimensions of 
educational opportunity by using standardized test score data from all public-school districts in 
the U.S. The first dimension was measured by the average achievement level among third grade 
students in the school district (i.e., average achievement), while the second dimension was 
measured by the rate of growth in academic achievement among students in the school district as 
they progressed from third to eighth grade (i.e., achievement growth). Reardon (2018) found that 
average achievement was highly correlated with a measure of community-wide socioeconomic 
status (r = 0.68), while achievement growth was only moderately correlated (r = 0.32). These 
correlations suggest that the two measures capture distinct dimensions of educational 
opportunity. While average achievement may reflect a broad range of community-wide 
educational opportunities available to students (e.g., neighborhood characteristics, average 


family resources in a community), achievement growth may better capture the unique 
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contributions of schooling toward promoting student achievement during the school-age years. 
Our study utilized Reardon’s (2018) framework to examine these dimensions of educational 
opportunity among children attending NC public schools—a school system where student 
achievement measures are commonly used for high-stakes accountability and quality monitoring. 
Following Reardon (2018), we also examined school districts rather than schools as the unit of 
analyses for educational opportunity. There are many ways in which school districts can directly 
and indirectly shape the quality of educational opportunity that students experience in their 
schools. 

Case studies demonstrate that school-district administrators make important decisions 
that directly influence the structural characteristics and culture of schools, including decisions 
about curriculum; decisions about hiring, staffing, and salaries of teachers and principals; as well 
as decisions about the professional development and evaluation of teachers and principals 
(Chenoweth, 2017, 2021). In extreme cases, school district administrators can fundamentally 
reform and/or reorganize schools through school closures, consolidations, and the 
implementation of accountability systems. School-district administrators can also influence the 
culture of school environments. For example, Chenoweth (2021) described effective school 
districts as having a culture in which the leadership cultivates the belief that all children are 
capable of success and that the adults are responsible for ensuring children succeed (p. 131). 
Indeed, a comprehensive case study of Chicago Public Schools identified effective leaders as the 
most critical ingredient in building a culture of school improvement (Bryk et al., 2010). In sum, 
school-district leadership can shape the structural organization and culture of schools, and, in 
turn, these features of the school environment can influence the quality of educational 


opportunity children experience (Eccles & Roeser, 2010). 
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The Current Study 

The quality of educational environments during school may serve to enhance or diminish 
the impacts of ECE as children progress through school. However, the extant evidence is mixed 
regarding if and how the quality of subsequent school environments will play a role in the 
persistence of ECE program impacts in the long run (Bailey, Jenkins, et al., 2020). The current 
study sought to address this issue by examining whether the long-term effects of North 
Carolina’s Smart Start and NC Pre-K programs were moderated by the quality of educational 
opportunity in NC public schools. In the first phase of our study, we used a growth curve model 
approach to index the quality of educational opportunity among students in NC public school 
districts by (1) the average level of academic achievement among third-grade students in the 
school district and (2) the rate of growth in academic achievement among students in the school 
district as they progressed from third to eighth grade.° In the second phase of our study, we 
considered variation in Smart Start and NC Pre-K program funding allocated to NC’s 100 
counties across 18 program years to examine the effects of program investments on student 
reading and mathematics outcomes in eighth grade—consistent with the analysis framework 
used in previous studies (Bai et al., 2020; Dodge et al., 2016; Ladd et al., 2014; Muschkin et al., 
2015).° We simultaneously tested for moderation of Smart Start and NC Pre-K program funding 
effects by the quality of educational opportunity in NC public schools. In particular, we tested 
alternative hypotheses in relation to the moderation of Smart Start and NC Pre-K program 


funding effects. Support for the dynamic complementarity hypothesis would be evidenced by 


> We considered average academic achievement level in third grade because it was the lowest grade level at which 
standardized achievement tests were administered in NC. 


® The current study extended the number of years of Smart Start and NC Pre-K program funding considered in the 
previous studies by six additional years to include 18 fiscal years of funding information. In doing so, we also 
increased the sample size by 303,803 students for the reading outcome analyses and 265,390 students for the math 
outcome analyses. 
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positive effects of program funding and educational opportunity as well as a positive interaction 
between these variables. Support for the dynamic substitutability hypothesis would be evidenced 
by positive effects of program funding and educational opportunity as well as a negative 
interaction between these variables. 
Methods 

Data Sources 

Our study used data on the full population of students enrolled in NC public schools in 
third grade through eighth grade between the 1996 and 2019 school years’ (school records 
obtained from the NC DPI) as well as a subpopulation of these students who were also born in 
NC between January 1, 1987, and August 31, 2005 (birth records obtained from NC State Center 
for Health Statistics). Data on the full population of students were used in the Phase I analyses 
while data on the subpopulation of students were used in the Phase IJ analyses (described in 
further detail in the Analyses section below). To combine these data, the identified birth records 
and school records of individual students were matched by the NC Education Research Data 
Center (NCERDC) at Duke University. In the current study, 74% of all birth records were 
matched to a school record. We also utilized on data on state allocations for Smart Start program 
funding obtained from the NC Partnership for Children, data on state allocations for NC Pre-K 
program funding obtained from the NC DPI, and publicly available data on each of NC’s 100 
counties obtained from the NC Office of State Budget and Management (OSBM). Institutional 
Review Board approval for this research was obtained at Duke University under the project titled 


“Combining Birth Data with Longitudinal Data on Schooling to Explore the Relationships 


T A school year is defined as July Ist of the previous year through June 30th of the listed year. 
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Between Children's Birth Weight, Immigrant Status, Pre-School Experiences and Performance in 
School” (Protocol: 2017-0495). 
Measures 
Smart Start funding exposure 

Data on Smart Start funding were obtained from the NC Partnership for Children. Smart 
Start was established during state fiscal year 1994 and funds were allocated to program 
contractors on a state fiscal year basis.* We converted all funds into 2019-dollar values based on 
the Consumer Price Index. To calculate exposure to Smart Start, we divided the amount of Smart 
Start funds awarded to each county during each fiscal year by the number of age-eligible 
children (1.e., ages 0-4) in each county during each fiscal year (based on county-level population 
estimates from NC OSBM).? To calculate cumulative exposure to Smart Start, we summed the 
annual Smart Start exposure in each county across each five-year period in which children in our 
panel were eligible to benefit from the program (i.e., prior to kindergarten entry between ages 0- 
4). After the program was established in 1994, the average county-level, cumulative exposure to 
Smart Start funding grew steadily and reached a peak of $2,383 in 2004, but then declined to 
$1,853 in 2010 (the last year of program funding considered in the current study) (see Figure 1). 
The average county-level, cumulative exposure to Smart Start funding between program years 


1994-2010 was $1,539. 


8 The NC Partnership for Children provided Smart Start funding data expended by program contractors in each of 
NC’s 100 counties during each fiscal year. 

° Note the Smart Start and NC Pre-K funding exposure variables were based on the entire population of age-eligible 
children even if only a portion of children received direct financial support to participate in the programs. 
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NC Pre-K funding exposure 

Data on NC Pre-K funding were obtained from the NC DPI. NC Pre-K was established 
during state fiscal year 2002 and funds were allocated to program contractors on a state fiscal 
year basis.!° We converted all funds into 2019-dollar values based on the Consumer Price Index. 
To calculate exposure to NC Pre-K, we divided the amount of NC Pre-K funds awarded to each 
county during each fiscal year by the number of age-eligible children (i.e., age 4) in each county 
during each fiscal year (based on county-level population estimates from NC OSBM). After the 
program was established in 2002, the average county-level exposure to NC Pre-K program 
funding grew steadily and reached a peak of $2,030 in 2010, with high variance within and 
across counties over the study period (see Figure 2). The average county-level exposure to NC 
Pre-K program funding between program years 2002-2010 was $1,064. We estimated that the 
program directly served 28% of all four-year-old children in NC in 2010 (the last year of 
program funding considered in the current study). 
Student Achievement in Reading and Mathematics 

Standardized assessments of reading and mathematics achievement were administered to 
students in NC public schools by school administrators at the end of each grade between third 
grade and eighth grade. Our study used scores from these end-of-grade (EOG) tests administered 
during all school years between 1996 and 2019.!! !* These data were obtained from the 


NCERDC. The EOG tests are state-mandated, standardized assessments with well-documented 


10 A total of 91 NC Pre-K program contractors served NC’s 100 counties during each fiscal year. Funds awarded to a 
single contractor that served multiple counties were disaggregated across counties within the contractor’s service 
region based on the population of age-eligible children in each county. 

'l The test scores of students who took alternative assessments designed for students with disabilities were excluded 
from these analyses because those test scales were not equated with the EOG scale. 

2 A number of advanced eighth grade students were not assessed with the EOG Math test in 2019, but were 
assessed using the high-school end-of-course (EOC) math test instead (1.6% of our input sample). These students 
were excluded from our analyses. 
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psychometric characteristics that satisfy the requirements of the U.S. Department of Education 
(Bazemore & Van Dyk, 2004; Mbella et al., 2016a, 2016b; North Carolina Department of Public 
Instruction, 2009; Sanford, 1996). 

Separate editions of the EOG tests were administered across the range of years included 
in our panel study (i.e., four editions of the EOG Reading and five editions of the EOG Math). 
To account for changes in the scaling of EOG test editions across years, we converted the EOG 
scores into Z-scores (across all editions) by subject, school year, and grade, resulting in a Z-score 
of 0.00 equal to the statewide average for students in each grade during each school year. 
Covariates 

Student characteristics. Data on individual students were obtained from (a) the birth 
records data and (b) the school records data. The student covariates include student sex (Female 
= 0, Male = 1; school records); student birth weight categorized as extremely low, very low, low, 
normal (reference category), or high (birth records); student race categorized as African 
American, Native American, Asian, Hispanic, mixed race, or non-Hispanic White (reference 
category) (school records); maternal education in years (birth records); parent marital status (Not 
married = 0, Married = 1; birth records); maternal age in years (birth records); no dad 
information on birth record (birth records); mother immigrant (No = 0, Yes = 1; birth records); 
student was first born (No = 0, Yes = 1; birth records); mother race categorized as African 
American, Native American, Asian, Hispanic, other race, or non-Hispanic White (reference 
category) (birth records); and the quarter of the school year in which the child was born 
categorized as quarter 1, 2, 3 (reference category; defined as the youngest children eligible for 


kindergarten entry during the school year), and 4. 
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School-district characteristics. Data on NC’s public-school districts across years were 
obtained from the NCERDC. The school-district covariates include the per pupil expenditures 
from local, state, and federal funding sources (converted to 2019 $s), the percent of economically 
disadvantaged students in the school district, and the total number of students in the school 
district’s membership. !3 

County characteristics. Data on each of NC’s 100 counties across years were obtained 
from (a) the NC OSBM data or (b) the birth records data aggregated from student-level 
information to county-level information. The county covariates include the percent of births to 
black mothers, Hispanic mothers, and low education mothers (< 12 years of education) (birth 
records); the percent of population receiving Food Stamps and the percent of population 
receiving Medicaid (NC OSBM); the total number of births (log transformed; birth records); the 
total population (log transformed; NC OSBM); and median family income (converted to 2019 
$s; NC OSBM). 

Analyses 

Analyses proceeded in two phases. In the first phase, we indexed the quality of 
educational opportunity in NC Public Schools by measuring school-district average achievement 
and achievement growth among students from third to eighth grade. Based on the resulting 
measures of average achievement and achievement growth, we examined associations of these 
measures. In the second phase of analysis, we examined the long-term effects of Smart Start and 
NC Pre-K program funding exposure on student achievement outcomes in eighth grade, and we 
tested whether these long-term effects were moderated by school-district average achievement 
and achievement growth. All analyses were completed in SAS® version 9.4. 


'3 Economic disadvantage was indexed based on the percent of students in the school district who qualified for free- 
or reduced-price lunch. 
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Phase I: Measuring School-District Average Achievement and Achievement Growth 

In Phase I of our analyses, we measured school-district average achievement and 
achievement growth for 18 cohorts of students who progressed from third to eighth grade in each 
of NC’s 115 public school districts (see Figure $1).!* !>-!© We used the reading and math scores 
drawn from the full population of students enrolled in NC public schools. The growth cohorts 
were defined as all students who took the third grade through eighth grade reading and/or math 
assessments during a six-year period.!7 !8 For example, the first growth cohort was defined by all 
students who took the third-grade reading and/or math assessment in 1996, the fourth-grade 
assessment in 1997, the fifth-grade assessment in 1998, the sixth-grade assessment in 1999, the 
seventh-grade assessment in 2000, and/or the eighth-grade assessment in 2001. 

We estimated a multi-level growth curve model for each of the 18 growth cohorts, with 
separate models estimated for reading and math scores (18 x 2 = 36 models total). All models 
were estimated using the HPMIXED procedure in SAS® version 9.4. The generalized equation 
for the multi-level growth curve models to measure school-district average achievement and 
achievement growth is shown below (with separate models calculated for each of the 19 


cohorts): 


4 We note that 4 public school districts were closed and consolidated with other public school districts during the 
course of our panel years. For our study, we aggregated information within these consolidated school districts prior 
to the years in which they were consolidated to utilize 115 school districts across all study years. 

5 All charter school districts were excluded from our study. The following public-school districts were also 
excluded from our study because they were not comparable to the 115 traditional public-school districts: (1) the NC 
Schools for the Blind and Deaf and (2) the NC School of Science and Mathematics. 

© There are 75 counties in NC that were served by a single school district, while 15 counties were served by two 
school districts. 

7 The scores of individual students who were retained in grade and had valid test scores recorded for the same grade 
assessment in separate years were included in the models for multiple cohorts (e.g., the scores for a student who 
took the third-grade assessment during the 1997 and 1998 school years were included in the models for cohorts 2 
and 3, respectively). 

'8 Within cohorts, the scores of students who changed school districts were included in each respective school 
district (e.g., the scores for a student who took the third- through fifth-grade assessments in school district A were 
nested within school district A, and the scores for that same student who took the six- through eighth-grade 
assessments in school district B were nested within school district B). 
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Level-1 (time; 2): 
Yeij = Moi + Wrijtimeri; + eri; 


Level-2 (students; i): 
Toij = Booj + ij 
N41ij = Bro; + ij 


Level-3 (school districts; j): 
Booj = Yooo + Uooj 
Boj = VY100 + U0; 


Variance Components: 
Ctij ~ N(O, Otij) 
Yoij ~ N(0, Ooij) 
Nij ~ NCO, o4i;) 
Uooj ~ N(O, Oo0j) 
U0; ~ N(0, O10;) 


Y,ij 1s the test score at time ¢ (grade) for student i in school district j; time,;; is coded as 0 at 
third grade, | at fourth grade, 2 at fifth grade, 3 at sixth grade, 4 at seventh grade, and 5 at 
eighth grade (separate models were calculated for reading and mathematics scores) 

Toi; is the expected achievement level (1.e., intercept) for student i in school district j in third 
grade 

7044; is the expected achievement growth (i.e., linear slope) for student i in school district j during 
each grade between third and eighth grade 

Boo; is the expected achievement level (i.e., intercept) for students in school district j in third 
grade 

B40; is the expected achievement growth (i.e., linear slope) for students in school district j during 
each grade between third and eighth grade 


In this model, repeated assessments of students’ reading or mathematics skills over time 
(t; Level-1) were nested within students (i; Level-2), and students were nested within school 
districts (7; Level-3). Time was coded as 0 at third grade, | at fourth grade, 2 at fifth grade, 3 at 
sixth grade, 4 at seventh grade, and 5 at eighth grade. Test scores (Y;;;) were modeled as a 
function of (1) an intercept term centered at the third-grade assessment score to represent the 
expected achievement level for student 7 in school district j in third grade (79;;) and the expected 


achievement level for students in school district j in third grade (B99 ;) as well as (2) a linear 


SMART START & NC PRE-K 22 


slope term to represent the expected achievement growth for student 7 in school district j during 
each grade between third and eighth grade (7, ;;) and the expected achievement growth for 
students in school district j during each grade between third and eighth grade (/19;). The 
intercept term was allowed to vary randomly between students (19; ;) and school districts (Ugo ;), 
and the slope term was also allowed to vary randomly between students (1,;;) and school districts 
(Uy9,;)- All of the variance terms were parameterized to be normally distributed random variables 
with a mean of zero. We specified an unstructured covariance matrix to allow the random 
intercepts and slopes and their variances to be correlated with one another at Level-2 and at 
Level-3 in order to allow for a systematic relation between intercepts and slopes. We used 
restricted maximum likelihood (REML) to estimate the variance components and the “residual” 
method to calculate denominator degrees of freedom. Students with at least one test score were 
included in these analyses and maximum likelihood was used to account for missing data (Singer 
& Willett, 2003). No covariates were included in these analyses in order to calculate 
unconditional estimates of the intercepts and slopes. Based on the results of this model, we 
derived estimates of average achievement (i.e., intercept; Bo9;) and achievement growth (i.e., 
slope; £1 ;) in reading/math for each of the 115 school-districts across the 18 cohorts for use in 
the subsequent analyses. Specifically, we derived the empirical best linear unbiased predictions 
(EBLUPs) for the realizations of the random intercept, slope, and nested errors. This model was 
conceptually similar to model 8.15—8.17 described by Bryk and Raudenbush (1992). Based on 
results of these analyses, we examined associations in order to better understand the validity of 
these measures, including associations between school-district economic disadvantage and 


average achievement/achievement growth. 
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Phase II: Eighth-Grade Student Achievement Analyses 
In Phase II of our analyses, we examined the effects of Smart Start and NC Pre-K 
program funding exposure on student reading and math achievement outcomes in eighth grade as 
well as moderation of these effects by school-district average achievement and achievement 
growth. Using two-way fixed effect regression analyses, we estimated program funding effects 
as the weighted average of between-county effects and within-county effects across time— 
weighted by variance and sample size (Kropko & Kubinec, 2020). A generalized equation is 
shown below for the two-way fixed effect analyses to examine moderation of Smart Start and 
NC Pre-K Program effects on students’ eighth grade outcomes by school-district average 
achievement and achievement growth: 
Oiscaypysy = Bo + Bi SS fundscpy + B2NCPK fundscpy 
+PzAvgACchscsy + BySS funds x AvgAchscsy + B5NCPK funds x AvgAChscsy 
+fAchGrowthgcsy + B7SS funds x AchGrowthgcsy + BgNCPK funds x AchGrowthgcsy 
+Xit Zecsy + Copy + Ae + Vpy + Ksy + @ * Kesy + Ciscpypysy 
In this model, O is the eighth-grade test score (with separate models estimated for reading 

and mathematics scores) for student 7, in school district s, in county c, born in birth year By, 
exposed to program funding during program year Py, and enrolled in eighth grade during school 
year Sy; 6, is the main effect of Smart Start funding exposure; fh is the main effect of NC Pre-K 
funding exposure; (3 is the main effect of school-district average achievement (i.e., average 
achievement in reading for the reading outcome model and average achievement in math for the 
math outcome model); {4 is the interaction between Smart Start funding exposure and school- 
district average achievement; f, is the interaction between NC-Pre-K funding exposure and 
school-district average achievement; f, is the main effect of school-district achievement growth 


(i.e., achievement growth in reading for the reading outcome model and achievement growth in 
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math for the math outcome model); £7 is the interaction between Smart Start funding exposure 
and school-district achievement growth; fg is the interaction between NC-Pre-K funding 
exposure and school-district achievement growth; X is a vector of effects for the student-level 
covariates; Z is a vector of effects for the school-district-level covariates; C is a vector of effects 
for the county-level covariates; @ is the fixed effect for county of residence to model between- 
county variation in program funding; y is the fixed effect for program year to model within- 
county variation in program funding over time; x is the fixed effect for school year to model 
variability in eighth-grade student achievement outcomes over time; and a * k is the linear time 
trend for counties across school years to account for possible preexisting time trends in student 
eighth-grade achievement outcomes within counties over time (for more details, see Vincent, 
2018) (see Figures S2 & S3 for county-level means of eighth grade reading and math Z-scores). 
We also examined if our findings were robust to the inclusion of quadratic time trends 

(@ * K * Kes). The program funding, county-level covariates, and county fixed effect were 
aligned to students’ county of residence at birth.!? *° The school-district average achievement 
and achievement growth measures, and the school district covariates were aligned to the 
students’ school district and school year during eighth grade. The models were estimated with 
robust standard errors clustered at the county level. The dependent variables and continuous 
independent variables were standardized for the analysis sample with M = 0, SD = 1 (with 


separate standardization conducted for the reading and math outcome analysis samples). 


'9 Tn previous studies, we have found the pattern of findings remained the same between analyses based on county of 
residence at birth and county of school attendance (Bai et al., 2020). 

20 We conducted a series of endogeneity analyses to examine the association between Smart Start & NC Pre-K 
program funding exposure and school-district average achievement & achievement growth (see Table S1). We 
found no reliable association between Smart Start & NC Pre-K funding and school district average achievement or 
achievement growth in reading and math. These findings suggest that our analyses satisfy a necessary condition to 
examine moderation. 
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Dichotomous independent variables were centered, but not standardized. All models were 
estimated using the SURVEYREG procedure in SAS® version 9.4. 

Based on the subpopulation of students with matched birth records and school records, 
we applied listwise deletion of cases with missing data on one or more study variables. From the 
matched sample, only 3% and 6% of students had missing data for the reading and math outcome 
analyses, respectively. For students who repeated eighth grade, we utilized only the first test 
score that was recorded during the first school year in which they were enrolled in that grade. 
Additionally, we implemented a “current year minus 1” approach to assign school-district 
average achievement and achievement growth values to students in a given school year on the 
basis of values derived from the prior school year, with a different set of students (e.g., students 
enrolled in eighth grade in 2002 were assigned school-district average achievement and 
achievement growth values from cohort #1, which was comprised of students enrolled in eighth 
grade in 2001, and so on).”! This approach was similar to that used by Andrabi et al. (2011). 

Results 

Table 1 summarizes the descriptive information on all study variables at the student level. 
Phase I: Measuring School-District Average Achievement and Achievement Growth 

We measured school-district average achievement and achievement growth in reading 
and mathematics for 18 cohorts of students who progressed from third to eighth grade between 
1996 and 2018. Based on EOG scores transformed to Z-scores, we found that school-district 


average achievement in third grade was —0.13 (SD = 0.22) for reading and —0.14 (SD = 0.23) for 


21 This approach was employed in order to avoid the confound using student test score information from the same 
students as both independent and dependent variables, because regressing student’s eighth grade achievement scores 
on school-district average achievement and achievement growth values measured from the same student test scores 
could introduce the same estimation errors on both the left- and right-hand side of the regression equation—resulting 
in biased estimates for school-district average achievement and achievement growth. 
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math (see Table S2). These findings indicated that the average third grade achievement of 
students in the public-school districts in our sample was slightly more than one tenth of a 
standard deviation below the statewide average.** Based on EOG scores transformed to Z-scores, 
we found that school-district achievement growth from third to eighth grade was —0.02 points 
(SD = 0.20) for reading and —0.01 points (SD = 0.20) for math.”> These findings indicated that, 
on average, students in the public-school districts in our sample showed only slight declines in 
their statewide rank-order standing in Z-scores between third and eighth grade. 

We then examined associations between school-district average 
achievement/achievement growth and school-district economic disadvantage. We found that 
school-district average achievement in reading and math was negatively associated with school- 
district economic disadvantage (PB =—0.46, p < .001; 6 =—0.39, p < .001) (see Table S3). 
Alternatively, we found that school-district achievement growth in reading and math was not 
reliably associated with school-district economic disadvantage (8 = —0.09, p = .21; B =—0.09, p 
= .26). Consistent with findings by Reardon (2018), these findings suggest that average 
achievement may reflect a broad range of socioeconomic conditions of the community where the 


school district is located, while achievement growth may better reflect the unique contributions 


>? Please note that students in charter school districts were included in the transformation of EOG scale scores to Z- 
scores, but these students were excluded from Phase I and II analyses. Therefore, a Z-score value of 0.00 indicates 
the statewide average for students in public school districts and charter school districts. The negative mean values 
for school-district average achievement and achievement growth imply that students in charter school districts 
scored higher than students in public school districts. 

3 Values for school-district achievement growth were rescaled to represent the rate of growth from third to eighth 
grade by multiplying the yearly rate of growth produced in the Phase I analyses by a value of 5. The interpretation 
for a school-district achievement growth score of 0.00 is that the school district’s rank-order standing remained at 
the statewide average from third to eighth grade. The interpretation for a positive school-district achievement growth 
score is that the school district gained in rank-order standing among all public-school districts from third to eighth 
grade, while the interpretation for a negative school-district achievement growth score is that the school district 
declined in rank-order standing among all public-school districts from third to eighth grade. 
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of schooling toward promoting student achievement during the elementary and middle school 
grades. 
Phase II: Eighth-Grade Student Achievement Analyses 

Table 2 summarizes the results of regression analyses to examine the effects of Smart 
Start and NC Pre-K funding exposure on students’ eighth-grade reading and math achievement 
as well as moderation of these program effects by each measure of school-district educational 
opportunity. Results are presented separately for student reading and math achievement 
outcomes. Within each outcome measure, separate results are presented for analyses with linear 
and quadratic time trends. 

Reading. We found positive effects of school-district average achievement in reading (6 
= 0.062, p < .001; Model 1) and achievement growth in reading (6 = 0.037, p < .001) on 
student’s eighth grade reading achievement. While the main effect of Smart Start funding 
exposure was not statistically significant (6 = 0.012, p = .097), the effect was moderated by 
school-district average achievement such that the magnitude of the Smart Start effect increased 
as school-district average achievement increased (8 = 0.007, p = .002). Moreover, when we 
probed the interaction, we found that the Smart Start effect was statistically significant at high 
levels of school-district average achievement (i.e., at +1 SD, B = 0.019, p = 0.01), but was not 
statistically significant at average (1.e., at the mean, 6 = 0.012, p = 0.10) and low levels of 
school-district average achievement (i.e., at —1 SD, B = 0.005, p = 0.53) (see Figure 3). We also 
found a positive main effect of NC Pre-K funding exposure (f = 0.017, p = .03) and the effect 
was moderated by school-district achievement growth such that the magnitude of the NC Pre-K 
effect decreased as school-district achievement growth increased (6 = —0.003, p = .04). 


Moreover, when we probed the interaction, we found that the NC Pre-K effect was not 
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statistically significant at high levels of school-district achievement growth (8 = 0.014, p = 0.08), 
but was statistically significant at average (6 = 0.017, p = 0.03) and low levels of school-district 
achievement growth (£ = 0.020, p = 0.01) (see Figure 4). These findings were robust to the 
inclusion of quadratic time trends. 

Math. We found positive effects of school-district average achievement in math (8 = 
0.083, p < .001) and achievement growth in math (B = 0.093, p < .001) on student’s eighth grade 
math achievement and these findings were robust to the inclusion of quadratic time trends. The 
main effects of Smart Start funding exposure (6 = 0.009, p = .29) and NC Pre-K funding 
exposure (6 = 0.020, p = .09) were not statistically significant and we found no evidence of 
moderation based on analyses with linear time trends. However, additional findings emerged 
with the inclusion of quadratic time trends. These findings include a positive main effect of 
Smart Start funding exposure (B = 0.035, p = .002) as well as moderation of the Smart Start 
effect by school-district average achievement such that the magnitude of the Smart Start effect 
increased as school-district average achievement increased ( = 0.006, p = .02) and moderation 
of the Smart Start effect by school-district achievement growth such that the magnitude of the 
Smart Start effect increased as school-district achievement growth increased (£6 = 0.007, p = .01). 
Additionally, the NC Pre-K effect was moderated by school-district average achievement such 
that the magnitude of the NC Pre-K effect decreased as school-district average achievement 
increased (6 = —0.005, p = .04). 

Discussion 

High-quality ECE can have long-term impacts on student educational outcomes 

throughout school and into adulthood (McCoy et al., 2017). However, what remains unclear is if 


and how these long-term impacts will be moderated by the quality of subsequent educational 
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environments during school (Bailey, Jenkins, et al., 2020). In the current study, we examined the 
eighth-grade effects of financial investments in NC’s two, flagship ECE programs—Smart Start 
and NC Pre-K—and we simultaneously examined moderation of these effects by the quality of 
educational opportunity in NC public school districts. We examined two hypotheses regarding 
the mechanisms by which subsequent education quality may moderate the long-term effects of 
ECE—dynamic complementarity and dynamic substitutability. Interestingly, we found support 
for dynamic complementarity in relation to the Smart Start program effect and we found support 
for dynamic substitutability in relation to the NC Pre-K program effect. 

With regard to the Smart Start effect on students’ eighth grade reading achievement, we 
found that increases in program funding exposure had a positive effect, but this effect was only 
evident in the context of higher-quality school districts—as measured by the average level of 
reading achievement among third-grade students in the school-district. This finding provides 
support for the dynamic complementarity hypothesis and suggests that Smart Start program 
effects are likely to be larger in the context of higher-quality schooling environments. 

With regard to the NC Pre-K effect on students’ eighth grade reading achievement, we 
found that increases in program funding exposure had a positive effect, and this effect was only 
evident in the context of average-quality and lower-quality school districts, but not higher- 
quality school districts—as measured by the rate of growth in academic achievement among 
students in the school district as they progressed from third to eighth grade. This finding provides 
support for the dynamic substitutability hypothesis and suggests that NC Pre-K program effects 
are likely to be smaller in the context of higher-quality schooling environments. It is important to 
note that this finding revealed that all students in high-growth school districts were more likely 


to attain higher reading achievement in eighth grade compared to students in lower-growth 
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school districts, regardless of their level of NC Pre-K program funding exposure. Therefore, the 
developmental benefit of higher-growth school districts appeared to act as a substitute for 
children with low or no exposure to NC Pre-K during early childhood. Simultaneously, higher 
exposure to NC Pre-K appeared to buffer children against the ill effect of subsequently enrolling 
in a lower-growth school district—the reading achievement of students in lower-growth school 
districts more closely approximated those of students in higher-growth school districts if they 
were also exposed to higher-levels of NC Pre-K program funding. 

Regarding the Smart Start and NC Pre-K effects on students’ eighth grade math 
achievement, we found evidence of program effects and moderation that was not robust to 
different analytic specifications (i.e., modeling linear or quadratic time trends). However, these 
additional findings provided further evidence of dynamic complementarity between Smart Start 
program exposure and subsequent educational opportunity as well as dynamic substitutability 
between NC Pre-K program exposure and subsequent educational opportunity. 

We speculate about the reasons why different dynamic effects emerged for Smart Start 
and NC Pre-K. First, scholars have suggested that dynamic substitutability is more likely to 
occur when ECE and subsequent schooling “are redundant in their content” (Bailey, Duncan, et 
al., 2020, p. 68). Dynamic substitutability could be evident for NC Pre-K because the program 
has long maintained a focus on direct instruction related to school readiness skills, including 
frequent and high-quality instruction related to early literacy building (e.g., Peisner-Feinberg et 
al., 2013). Moreover, observations of pre-kindergarten classrooms across 11 states (including 
data from NC) has also confirmed that language/literacy instruction is the most frequent learning 
activity observed in pre-kindergarten classrooms, constituting an estimated 17% of the entire day 


(Early et al., 2010). Hence, in communities with low or no exposure to NC Pre-K, enrolling in a 
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school district with higher rates of growth in reading achievement may have enabled students to 
make up the gains in underlying literacy skills that NC Pre-K would have otherwise conferred. 

Alternatively, dynamic complementarity could be evident for Smart Start because the 
program’s goals were related to broad improvements in the fundamental infrastructure of ECE— 
including child care affordability, availability, and quality, as well as family support and child 
health—rather than an explicit focus on direct instruction related to early literacy skills, akin to 
the focus provided by NC Pre-K. Nonetheless, Smart Start’s broad mission may have served to 
promote student literacy achievement in the long run through means other than direct instruction 
in early literacy skills, such as enhanced family engagement in book reading. Therefore, in 
communities with high exposure to Smart Start, enrolling in a school district with higher levels 
of reading achievement may have enabled students to further build on the broad developmental 
benefits conferred through Smart Start. In sum, it is possible that we observed dynamic 
substitutability for NC Pre-K because the benefits conferred through its specific intervention 
components (e.g., early literacy instruction) closely corresponded to the benefits conferred 
through literacy instruction in high-growth school districts. In contrast, the specific intervention 
components characteristic of Smart Start programming might confer benefits that lay a broader 
foundation for high-quality schooling to build on, so we see dynamic complementarity. 

In the debate about fadeout or persistence of ECE program effects on student 
achievement, the findings of our research should encourage educational stakeholders and 
policymakers to consider how subsequent schooling environments play a role in determining 
whether program effects are maintained or diminished in the long run. While previous studies 
have documented the enduring effects of Smart Start and NC Pre-K programming through 


elementary school and middle school (Bai et al., 2020; Dodge et al., 2016; Ladd et al., 2014; 
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Muschkin et al., 2015), findings from the current study point out the conditional nature of these 
effects. We found that the quality of educational opportunity during school made a difference in 
determining whether ECE program effects on student achievement were maintained or 
diminished in the long run. Moreover, we found that the conditions were varied—including 
evidence of dynamic complementarity between Smart Start and subsequent educational 
opportunity as well as evidence of dynamic substitutability between NC Pre-K and subsequent 
educational opportunity. Today, educators and policymakers recognize the importance of 
promoting continuity in high-quality education across the lifespan, and several frameworks have 
been developed to facilitate educational alignment (e.g., Kauerz & Coffman, 2013; Stipek et al., 
2017; Zellman & Kilburn, 2015). Consistent with these alignment frameworks, we advocate for a 
view of ECE programming and its benefits as part of a continuum of high-quality educational 
inputs to be built upon sequentially across development (Carr, 2021). 

Our study was unique in focusing on the quality of educational opportunity in school 
districts as the unit of analysis. We found that school-district average achievement and 
achievement growth both had positive main effects on student reading and math achievement in 
eighth grade. However, it is important to note that average achievement—but not achievement 
growth—showed a reliable, negative association with a measure of school-district 
socioeconomic disadvantage. This suggests that average achievement may index a wide range of 
educational opportunities available to students in their communities. Alternatively, achievement 
growth may better reflect the unique contributions of schooling toward promoting student skill 
development. Therefore, these different dimensions of educational opportunity may represent 
different pathways toward raising student achievement. For example, the benefits of enrolling in 


a school district with higher levels of average achievement may begin to accrue as soon as 
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children step inside the kindergarten door—impacting early skill development—while the 
benefits of achievement growth may require multiple years of exposure to accrue benefits— 
impacting later skill development. Additionally, the educational benefits of enrolling in a school 
district with higher levels of average achievement may extend beyond the experiences students 
have during the school day to their experiences in community centers and after school programs. 

Finally, although the effects found in this study may be considered small by conventional 
standards (e.g., a 0.017 SD increase in eighth grade reading scores with every 1 SD unit increase 
in NC Pre-K funding exposure), these effects were educationally meaningful according to 
empirical benchmarks established for educational interventions (Kraft, 2020). These small 
magnitude effects can be further contextualized by the moderate cost (1.e., not small or large; 
Kraft, 2020) of the average level of Smart Start and NC Pre-K program exposure ($1,539 and 
$1,064, respectively). Overall, reasonable cost investments made to the Smart Start and NC Pre- 
K programs produced population-wide improvements in student achievement outcomes that 
remained educationally meaningful in the long-term—suggesting that even larger investments 
could produce greater impact in the future. 
Study Limitations 

The current study focused on school districts as the unit of analyses to measure the 
quality of educational opportunity, and our measures of average achievement and achievement 
growth were both positively associated with student achievement in eighth grade. Nonetheless, it 
is possible that the quality of educational opportunity may vary between schools within school 
districts. However, this consideration was outside the scope of our study because many schools 
did not include the full range of grade levels needed to measure average achievement and 


achievement growth from third to eighth grade. Additionally, we did not examine moderation of 
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ECE program exposure effects in earlier grades, because school-district average achievement and 
achievement growth entailed measurement of student achievement between third and eighth 
grade—confounding earlier grade outcomes with information from eighth grade academic 
achievement. 
Conclusion 

Our study adds to a growing body of evidence documenting the population-wide 
improvements in student achievement resulting from financial investments in ECE programs 
(e.g., Fitzpatrick, 2008; Kose, 2021). We confirmed prior research to indicate that financial 
investments NC’s two, flagship ECE programs—Smart Start and NC Pre-K—can have 
population-wide effects on student academic achievement that endure through the end of middle 
school—nine years after program exposure (Bai et al., 2020). The current study also contributes 
new evidence regarding the role of school-age educational opportunity in differentiating these 
long-term effects. Specifically, we found that the positive effect of financial investments in 
Smart Start was larger for students who subsequently enrolled in higher-quality school districts, 
while the positive effect of NC Pre-K was smaller for students who subsequently enrolled in 
higher-quality school districts. The existing literature in this area was quite mixed to begin with 
(see Bailey, Jenkins, et al., 2020), and our findings provide support for two alternative 
hypotheses—including dynamic complementarity and dynamic substitutability—as mechanisms 
by which the quality of subsequent educational opportunity during school may moderate the 
long-term benefits of ECE programming. In the debate about fadeout or persistence of ECE 
program effects on student achievement, we encourage educational stakeholders and 
policymakers to consider how subsequent schooling environments can play a role in determining 


whether program effects are maintained or diminished in the long run. 
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Descriptive Statistics for the Eighth Grade Student Reading Achievement Analysis Sample (N = 


1,164,948) 


Variable M (SD) 
Program funding 
Smart Start funding exposure (in $00’s; all study years) 14.56 (10.90) 
NC Pre-K funding exposure (in $00’s; all study years) 4.18 (6.71) 
Eighth grade achievement outcomes 
Reading -0.01 (1.00) 
Math 0.00 (1.00) 
School-district average achievement 
Reading -0.09 (0.18) 
Math -0.10 (0.19) 
School-district achievement growth 
Reading -0.02 (0.10) 
Math 0.00 (0.16) 
Student characteristics 
Sex (Male) 0.50 (0.50) 
Extremely low birth weight 0.00 (0.06) 
Very low birth weight 0.01 (0.09) 
Low birth weight 0.07 (0.25) 
Normal birth weight 0.82 (0.38) 
High birth weight 0.10 (0.29) 
Non-Hispanic White 0.58 (0.49) 
African American 0.29 (0.45) 
Native American 0.02 (0.13) 
Asian 0.01 (0.11) 
Hispanic 0.07 (0.25) 
Mixed race 0.03 (0.17) 
Maternal education (years) 12.58 (2.55) 
Parent marital status 0.65 (0.48) 
Maternal age (years) 26.11 (5.93) 
No dad information 0.14 (0.34) 
Mother immigrant 0.09 (0.29) 
First born 0.43 (0.50) 
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Descriptive Statistics for the Eighth Grade Student Reading Achievement Analysis Sample (N = 


1,164,948) 
Variable M (SD) 
Mother White 0.68 (0.47) 
Mother African American 0.29 (0.45) 
Mother Native American 0.02 (0.12) 
Mother Asian 0.01 (0.12) 
Mother Hispanic 0.07 (0.26) 
Mother other race 0.00 (0.03) 
School-district characteristics, eighth-grade year 
Total per pupil expenditures (2019 $s) 9821 (976) 
Economic disadvantage (%) 52.00 (15.76) 
School district membership 39653 (45491) 
County characteristics, birth year 
Births to black mothers (% of births) 25.75 (14.46) 
Births to Hispanic mothers (% of births) 8.03 (11.37) 
Births to low education mothers (% of births) 22.59 (6.56) 
Population on Food Stamps (% of population) 7.40 (3.64) 
Population on Medicaid (% of population) 14.40 (5.90) 
Number of births (log) 7.51 (1.09) 
Total population (log) 11.77 (1.00) 
Median family income (2019 $s) 68289 (13223) 


Note. Descriptive statistics are provided for those students included in the reading outcome 
analyses (N = 1,164,948). Descriptive statistics for students included in the math outcome 


analyses are provided in Table S4. The Smart Start and NC Pre-K funding variables were scaled 
such that a value of 1.00 corresponds to $100 in funding. School-district average achievement in 
reading/math (B99 ;) was measured at third grade and achievement growth in reading/math (4 9,) 


was measured from third to eighth grade. All values of school-district average achievement and 
achievement growth were “current year minus 1.” Economic disadvantage was indexed based on 


the percent of students in the school district who qualified for free- or reduced-price lunch 
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Reading Math 
N = 1,164,948 N = 1,127,587 
Parameter B (SE) B (SE) B (SE) B (SE) 
Smart Start funding exposure 0.012" (0.007) 0.015° (0.006) 0.009 (0.008) 0.035" (0.011) 
NC Pre-K funding exposure 0.017* (0.008) 0.021™ (0.008) 0.0207 (0.012) —0.003 (0.010) 
Average Achievement 0.062°*" (0.005) 0.061°*" (0.005) 0.083°** (0.005) 0.081°*" (0.005) 
Smart Start x Average Achievement 0.007" (0.002) 0.009**™" (0.002) 0.003 (0.003) 0.006" (0.002) 
NC Pre-K x Average Achievement —0.003 (0.002) ~0.004" (0.002) 0.001 (0.003) ~0.005° (0.002) 
Achievement Growth 0.037" (0.003) 0.036" (0.003) 0.093"" (0.004) 0.089"" (0.004) 
Smart Start x Achievement Growth 0.003 (0.002) 0.002 (0.002) 0.006" (0.003) 0.007" (0.003) 
NC Pre-K x Achievement Growth —0.003° (0.001) ~0.005™" (0.001) 0.003 (0.003) —0.001 (0.003) 
Student, school-district, & county covariates Inc. Inc. Inc. Inc. 
County fixed effects Inc. Inc. Inc. Inc. 
Program year fixed effects Inc. Inc. Inc. Inc. 
School year fixed effects Inc. Inc. Inc. Inc. 
County/school year linear time trends Inc. Inc. Inc. Inc. 
County/school year quadratic time trends Inc. Inc. 
R-squared 0.29 0.30 0.29 0.30 


Note. Parameter estimates for the student, school-district, and county covariates; county fixed effects; program year fixed effects; 
school year fixed effects; and county/school year linear and quadratic time trends are not displayed in Table 2, but can be made 
available from the corresponding author upon request. School-district average achievement/achievement growth correspond to reading 
average achievement/achievement growth in the reading models and math average achievement/achievement growth in the math 
models. All values of school-district average achievement and achievement growth were “current year minus 1.” The student outcome 
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variables, the Smart Start and NC Pre-K funding exposure variables, and the school-district average achievement and achievement 
growth variables were standardized with M = 0, SD = 1. All models were estimated with robust standard errors clustered at the county 
level. ‘p < 0.10, “p < 0.05, ““p < 0.01, “*p < 0.001 
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Figure 1 


Five Year Sum of Smart Start Funds per O- to 4-Year-Old Children in County: 1993 to 2010 


$5,000 


$4,000 


$3,000 


$2,000 


Five Year Sum of Smart Start Funds 


$1,000 


per 0- to 4-Year-Old Children in County in 2019 $s 


State Fiscal Years 


Note. All counties with statewide average. Dotted lines represent individual counties. Solid line 
represents statewide average. Smart Start was established in state fiscal year 1994. In the student 
outcome analyses, all counties were assigned a pre-trend value of “0” prior to the establishment 
of Smart Start during state fiscal year 1994. 
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Figure 2 


NC Pre-K Funds per 4-Year-Old Children in County: 1993 to 2010 
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Note. All counties with statewide average. Dotted lines represent individual counties. Solid line 
represents statewide average. NC Pre-K was established in state fiscal year 2002. In the student 
outcome analyses, all counties were assigned a pre-trend value of “0” prior to the establishment 
of NC Pre-K in state fiscal year 2002. 
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Figure 3 


Moderation of the Smart Start Funding Effect on Eighth Grade Student Reading Achievement by 
School-District Average Achievement in Reading 
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Note. AA = Average achievement. The effect of Smart Start funding exposure on student reading 
outcomes in eighth grade at three values of school-district average achievement in reading: | SD 
below the average (low), at the average, and 1 SD above the average (high). 
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Figure 4 


Moderation of the NC Pre-K Funding Effect on Eighth Grade Student Reading Achievement by 
School-District Achievement Growth in Reading 
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Note. AG = Achievement growth. The effect of NC Pre-K funding exposure on student reading 
outcomes in eighth grade at three values of school-district achievement growth in reading: 1 SD 
below the average (low), at the average, and 1 SD above the average (high). 
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Table S1 


Analyses to Examine the Association between Program Funding Exposure and School-District Average Achievement & Achievement 
Growth (N = 2,070) 


Reading Math 
Average Achievement Average Achievement 
Achievement Growth Achievement Growth 
Parameter B (SD) B (SD) B (SD) B (SD) 
Smart Start funding exposure 0.05 (0.04) —0.12 (0.08) 0.03 (0.05) —0.11 (0.08) 
NC Pre-K funding exposure 0.01 (0.03) 0.11 (0.08) —0.08 (0.05) 0.14 (0.09) 


Note. The generalized equation for these analyses was: 
Oscpy = Bo + By Smart Start funding. py + B,NC PreK funding-py + Ae +Ypy + Escpy 


O is the school-district average achievement or achievement growth score for school district s in county c aligned to program year Py, 
fp, is the main effect of Smart Start funding, fz is the main effect of NC Pre-K funding, a is the fixed effect for county, and y is the 
fixed effect for program year. County-level program funding information in program year T was aligned with school-district average 
achievement and achievement growth information during school year T + 4 (e.g., students’ exposure to program funding in pre-K 
during program year 2000 was aligned to the average achievement and achievement growth values for students in third grade four 
years later during school year 2004, which is the school year in which these students were expected to be in third grade). Additionally, 
all values of school-district average achievement and achievement growth were “current year minus 1.” The Smart Start and NC Pre-K 
funding variables as well as the school-district average achievement and achievement growth variables were standardized with M = 0, 
SD = 1. All models were estimated with robust standard errors clustered at the county level. “p < 0.05, ““‘p < 0.01, ““p < 0.001 
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Table S2 


Descriptive Statistics for School-District Average Achievement and Achievement Growth Presented at the School-District Level (N = 


2,070) 
Variable M 
School-district average achievement 
Reading -0.13 
Math -0.14 
School-district achievement growth 
Reading -0.02 
Math -0.01 


(SD) 


(0.22) 
(0.23) 


(0.12) 
(0.20) 


Note. School-district average achievement in reading/math (fo ;) was measured at third grade and achievement growth in 
reading/math ((j9;) was measured from third to eighth grade. Economic disadvantage was indexed based on the percent of students in 
the school district who qualified for free- or reduced-price lunch. There were 115 school districts in each of the 18 cohorts (1.e., 18 x 


15 = 2,070 school district by cohort observations). 
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Table S3 


Analyses to Examine the Association between School-District Economic Disadvantage and School-District Average Achievement & 
Achievement Growth (N = 2,070) 


Reading Math 
Average Achievement Average ~—~—sO Achievement 
Achievement Growth Achievement Growth 
Parameter B (SD) B (SD) B (SD) B (SD) 
Economic disadvantage (%) —0.46"" (0.08) —0.09 (0.07) —0.39"™" — (0.10) —0.09 (0.08) 


Note. Economic disadvantage is the percent of economically disadvantaged students in the school district (i.e., the percent of students 
in the school district who qualified for free- or reduced-price lunch). The generalized equation for these analyses was: 


Oscsy = Bo + Bi Economic Disadvantagescsy + Ac +¥sy + Escsy 


O is the school-district average achievement or achievement growth score for school district s in county c aligned to school year Sy, /, 
is the main effect of economic disadvantage, a is the fixed effect for county, and y is the fixed effect for school year. School-district 
economic disadvantage information in school year Sy was aligned with school-district average achievement and achievement growth 
information for the cohort of students in third grade during school year Sy (e.g., school-district economic disadvantage in school year 
2000 was aligned to the average achievement and achievement growth values for the cohort of students in third grade in 2000). The 
school-district economic disadvantage, average achievement, and achievement growth variables were standardized with M = 0, SD = 
1. All models were estimated with robust standard errors clustered at the county level. “p < 0.05, “‘p < 0.01, “"p < 0.001 
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Table S4 


Descriptive Statistics for the Eighth Grade Student Math Achievement Analysis Sample (N = 
1,127,587) 


Variable M (SD) 
Program funding 
Smart Start funding exposure (in $00’s; all study years) 14.46 (11.05) 
NC Pre-K funding exposure (in $00’s; all study years) 3.84 (6.39) 


Eighth grade achievement outcomes 
Reading -0.04 (1.00) 
Math 0.00 (1.00) 


School-district average achievement 
Reading -0.09 (0.18) 
Math -0.10 (0.19) 


School-district achievement growth 
Reading -0.02 (0.10) 
Math -0.01 (0.16) 


Student characteristics 


Sex (Male) 0.50 (0.50) 
Extremely low birth weight 0.00 (0.06) 
Very low birth weight 0.01 (0.09) 
Low birth weight 0.07 (0.25) 
Normal birth weight 0.82 (0.38) 
High birth weight 0.10 (0.29) 
Non-Hispanic White 0.58 (0.49) 
African American 0.30 (0.46) 
Native American 0.02 (0.13) 
Asian 0.01 (0.10) 
Hispanic 0.07 (0.25) 
Mixed race 0.03 (0.17) 
Maternal education (years) 12.53 (2.53) 
Parent marital status 0.65 (0.48) 
Maternal age (years) 26.01 (5.91) 
No dad information 0.14 (0.35) 
Mother immigrant 0.09 (0.28) 


First born 0.43 (0.50) 
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Table S4 (continued) 


Descriptive Statistics for the Eighth Grade Student Math Achievement Analysis Sample (N = 
1,127,587) 


Variable M (SD) 
Mother White 0.67 (0.47) 
Mother African American 0.30 (0.46) 
Mother Native American 0.02 (0.13) 
Mother Asian 0.01 (0.11) 
Mother Hispanic 0.07 (0.26) 
Mother other race 0.00 (0.03) 


School-district characteristics, eighth-grade year 


Total per pupil expenditures (2019 $s) 9817 (978) 
Economic disadvantage (%) 51.88 (15.64) 
School district membership 38842 (44607) 


County characteristics, birth year 


Births to black mothers (% of births) 25.85 (14.53) 
Births to Hispanic mothers (% of births) Ted (11.43) 
Births to low education mothers (% of births) 22.60 (6.58) 
Population on Food Stamps (% of population) 7.38 (3.65) 
Population on Medicaid (% of population) 14.32 (5.90) 
Number of births (log) TA9 (1.08) 
Total population (log) 11.75 (1.00) 
Median family income (2019 $s) 67998 (13132) 


Note. Descriptive statistics are provided for those students included in the math outcome 
analyses (N = 1,127,587). The Smart Start and NC Pre-K funding variables were scaled such that 
a value of 1.00 corresponds to $100 in funding. School-district average achievement in 
reading/math (P99 ;) was measured at third grade and achievement growth in reading/math (fj 9;) 
was measured from third to eighth grade. All values of school-district average achievement and 
achievement growth were “current year minus 1.” Economic disadvantage was indexed based on 
the percent of students in the school district who qualified for free- or reduced-price lunch 
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Figure S1 
North Carolina Third to Eighth Grade Student Cohorts between 1996 and 2018 (N = 18) 
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Note. Each diagonal line corresponds to a cohort of students. In Phase I of our analyses, a separate growth curve model was estimated 
for students in each cohort, which was used to calculate the intercept (i.e., third grade average achievement) and slope (_.e., 
achievement growth between third and eighth grade) in reading and mathematics for each school district within each cohort. These 
estimates were then used in Phase II of our analyses to examine moderation of the long-term effects of Smart Start and NC Pre-K. 
Students in the cohort indexed by the asterisk were included in the Phase II analyses, but not the Phase I analyses because of the 
“current year minus 1” approach. 
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Figure S2 
County-Level Means of Eighth-Grade Reading Z-scores: School Years 2002-2019 
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Note. All 100 counties with statewide average. Dotted lines represent the individual counties. 
Solid line represents statewide average. A smoothing function (the generalized additive model) 
was applied to the data. 
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Figure S3 
County-Level Means of Eighth-Grade Math Z-scores: School Years 2002-2019 
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Note. All 100 counties with statewide average. Dotted lines represent the individual counties. 
Solid line represents statewide average. A smoothing function (the generalized additive model) 


was applied to the data. 


